Face Generation

In this project, you'll define and train a DCGAN on a dataset of faces. Your goal is to get a generator network to generate new images of faces that look as realistic as possible!

The project will be broken down into a series of tasks from loading in data to defining and training adversarial networks. At the end of the notebook, you'll be able to visualize the results of your trained Generator to see how it performs; your generated samples should look like fairly realistic faces with small amounts of noise.

Get the Data

You'll be using the CelebFaces Attributes Dataset (CelebA) to train your adversarial networks.

This dataset is more complex than the number datasets (like MNIST or SVHN) you've been working with, and so, you should prepare to define deeper networks and train them for a longer time to get good results. It is suggested that you utilize a GPU for training.

Pre-processed Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. Some sample data is show below.

If you are working locally, you can download this data by clicking here

This is a zip file that you'll need to extract in the home directory of this notebook for further loading and processing. After extracting the data, you should be left with a directory of data processed_celeba_small/

In [1]:
data_dir = 'images/'

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np
import problem_unittests as tests
#import helper

%matplotlib inline

Visualize the CelebA Data

The CelebA dataset contains over 200,000 celebrity images with annotations. Since you're going to be generating faces, you won't need the annotations, you'll only need the images. Note that these are color images with 3 color channels (RGB)#RGB_Images) each.

Pre-process and Load the Data

Since the project's main focus is on building the GANs, we've done some of the pre-processing for you. Each of the CelebA images has been cropped to remove parts of the image that don't include a face, then resized down to 64x64x3 NumPy images. This pre-processed dataset is a smaller subset of the very large CelebA data.

There are a few other steps that you'll need to transform this data and create a DataLoader.

Exercise: Complete the following get_dataloader function, such that it satisfies these requirements:

  • Your images should be square, Tensor images of size image_size x image_size in the x and y dimension.
  • Your function should return a DataLoader that shuffles and batches these Tensor images.

ImageFolder

To create a dataset given a directory of images, it's recommended that you use PyTorch's ImageFolder wrapper, with a root directory processed_celeba_small/ and data transformation passed in.

Data Citation

@inproceedings{liu2015faceattributes, title = {Deep Learning Face Attributes in the Wild}, author = {Liu, Ziwei and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou}, booktitle = {Proceedings of International Conference on Computer Vision (ICCV)}, month = {December}, year = {2015} }

In [2]:
# necessary imports
import torch
from torchvision import datasets
from torchvision import transforms
from torch.utils.data import DataLoader
In [3]:
def get_dataloader(batch_size, image_size, data_dir='images/'):
    """
    Batch the neural network data using DataLoader
    :param batch_size: The size of each batch; the number of images in a batch
    :param img_size: The square size of the image data (x, y)
    :param data_dir: Directory where image data is located
    :return: DataLoader with batched data
    """
    
    # TODO: Implement function and return a dataloader
    transformation = transforms.Compose([transforms.Resize(image_size),
                                     transforms.CenterCrop(image_size),
                                     transforms.ToTensor()])
    data = datasets.ImageFolder(data_dir,transform=transformation)
    loader = DataLoader(data,batch_size=batch_size,shuffle= True)
    return loader

Create a DataLoader

Exercise: Create a DataLoader celeba_train_loader with appropriate hyperparameters.

Call the above function and create a dataloader to view images.

  • You can decide on any reasonable batch_size parameter
  • Your image_size must be 32. Resizing the data to a smaller size will make for faster training, while still creating convincing images of faces!
In [4]:
# Define function hyperparameters
batch_size = 32
img_size = 32

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# Call your function and get a dataloader
celeba_train_loader = get_dataloader(batch_size, img_size)

Next, you can view some images! You should seen square images of somewhat-centered faces.

Note: You'll need to convert the Tensor images into a NumPy type and transpose the dimensions to correctly display an image, suggested imshow code is below, but it may not be perfect.

In [5]:
# helper display function
def imshow(img):
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# obtain one batch of training images
dataiter = iter(celeba_train_loader)
images, _ = dataiter.next() # _ for no labels

# plot the images in the batch, along with the corresponding labels
fig = plt.figure(figsize=(20, 4))
plot_size=20
for idx in np.arange(plot_size):
    ax = fig.add_subplot(2, plot_size/2, idx+1, xticks=[], yticks=[])
    imshow(images[idx])

Exercise: Pre-process your image data and scale it to a pixel range of -1 to 1

You need to do a bit of pre-processing; you know that the output of a tanh activated generator will contain pixel values in a range from -1 to 1, and so, we need to rescale our training images to a range of -1 to 1. (Right now, they are in a range from 0-1.)

In [6]:
# TODO: Complete the scale function
def scale(x, feature_range=(-1, 1)):
    ''' Scale takes in an image x and returns that image, scaled
       with a feature_range of pixel values from -1 to 1. 
       This function assumes that the input x is already scaled from 0-1.'''
    # assume x is scaled to (0, 1)
    # scale to feature_range and return scaled x
    min,max = feature_range
    x = x * (max-min) + min
    return x
In [7]:
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
# check scaled range
# should be close to -1 to 1
img = images[0]
scaled_img = scale(img)

print('Min: ', scaled_img.min())
print('Max: ', scaled_img.max())
Min:  tensor(-0.9843)
Max:  tensor(0.9686)

Define the Model

A GAN is comprised of two adversarial networks, a discriminator and a generator.

Discriminator

Your first task will be to define the discriminator. This is a convolutional classifier like you've built before, only without any maxpooling layers. To deal with this complex data, it's suggested you use a deep network with normalization. You are also allowed to create any helper functions that may be useful.

Exercise: Complete the Discriminator class

  • The inputs to the discriminator are 32x32x3 tensor images
  • The output should be a single value that will indicate whether a given image is real or fake
In [8]:
import torch.nn as nn
import torch.nn.functional as F
from torch.nn.utils import spectral_norm
In [29]:
class Discriminator(nn.Module):
    
    def __init__(self, conv_dim):
        """
        Initialize the Discriminator Module
        :param conv_dim: The depth of the first convolutional layer
        """
        super(Discriminator, self).__init__()

        # complete init function
        self.conv_dim = conv_dim
        #define layers
        #first conv layer reducing dimensionality by 2 and increasing conv_dim to conv_dim
        self.conv1 = nn.Conv2d(3,conv_dim,4,padding=1,stride=2)
        #second conv layer reducing dimensionality by 2 and increasing conv_dim to 2
        self.conv2 = nn.Conv2d(conv_dim , conv_dim*2,4,padding=1,stride=2)
        #instance norm after 2nd conv layer to increase model performance
        self.in2 = nn.InstanceNorm2d(conv_dim*2)
 
        #third conv layer reducing dimensionality by 2 and increasing conv_dim to 4
        self.conv3 = nn.Conv2d(conv_dim*2,conv_dim*4,4,padding=1,stride=2)
        #instance norm after 3rd conv layer to increase model performance
        self.in3 = nn.InstanceNorm2d(conv_dim*4)

        #final linear layer with 1 output, BCEloss will be calculated so no need to apply sigmoid.
        self.fcout = nn.Linear(conv_dim*8*8,1)
        #1d and 2d dropout layers to improve results.
        self.dropout = nn.Dropout2d(p=0.3)
        self.dropout1d = nn.Dropout(p=0.3)
        
        

    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: Discriminator logits; the output of the neural network
        """
        # define feedforward behavior
        #pass input tensor thru network
        x = F.leaky_relu(self.conv1(x))
        x = self.dropout(x)
        x = self.conv2(x)
        x = F.leaky_relu(self.in2(x))
        x = self.dropout(x)
        x = self.conv3(x)
        x = F.leaky_relu(self.in3(x))
        x = self.dropout(x)
        #reshape vector to fit into linear layer
        x = x.view(-1,self.conv_dim*8*8)
        x = self.dropout1d(x)
        out = self.fcout(x)
        #return logit of  linear layer
        return out


"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_discriminator(Discriminator)
Tests Passed

Generator

The generator should upsample an input and generate a new image of the same size as our training data 32x32x3. This should be mostly transpose convolutional layers with normalization applied to the outputs.

Exercise: Complete the Generator class

  • The inputs to the generator are vectors of some length z_size
  • The output should be a image of shape 32x32x3
In [30]:
class Generator(nn.Module):
    
    def __init__(self, z_size, conv_dim):
        """
        Initialize the Generator Module
        :param z_size: The length of the input latent vector, z
        :param conv_dim: The depth of the inputs to the *last* transpose convolutional layer
        """
        super(Generator, self).__init__()

        # complete init function
        self.z_size = z_size
        self.conv_dim = conv_dim
        #define layers
        #dense layer reshapping fron z_size to conv_dim*16
        self.fcin = nn.Linear(z_size,conv_dim*16)
        #first conv tranpose layer increasing dimensionality by 2 and decraising conv_dim to 8
        self.deconv1 = nn.ConvTranspose2d(conv_dim*16,conv_dim*8,4,padding=1,stride=2)
        #instance norm after 1st deconv layer to increase model performance
        self.in1 = nn.InstanceNorm2d(conv_dim*8)

        #second convtranspose  layer increasing dimensionality by 2 and decreasing conv_dim to 4
        self.deconv2 = nn.ConvTranspose2d(conv_dim*8,conv_dim*4,4,padding=1,stride=2)
        #instance norm after 2nd deconv layer to increase model performance
        self.in2 = nn.InstanceNorm2d(conv_dim*4)

        #third conv transpose layer increasing dimensionality by 2 and decreasing conv_dim to 2
        self.deconv3 = nn.ConvTranspose2d(conv_dim*4,conv_dim*2,4,padding=1,stride=2)
        #instance norm after 3rd deconv layer to increase model performance
        self.in3 = nn.InstanceNorm2d(conv_dim*2)


        #fourth conv transpose layer increasing dimensionality by 2 and decreasing conv_dim to conv_dim
        self.deconv4 = nn.ConvTranspose2d(conv_dim*2 , conv_dim,4,padding=1,stride=2)
        #instance norm after 4th conv layer to increase model performance
        self.in4 = nn.InstanceNorm2d(conv_dim)

        #last conv transpose layer increasing dimensionality by 2 and increasing conv_dim to 3
        self.deconv5 = nn.ConvTranspose2d(conv_dim,3,4,padding=1,stride=2)
        #put in dropout layers to improve results
        self.dropout1d = nn.Dropout(p=0.3)
        self.dropout2d = nn.Dropout2d(p=0.3)

    def forward(self, x):
        """
        Forward propagation of the neural network
        :param x: The input to the neural network     
        :return: A 32x32x3 Tensor image as output
        """
        # define feedforward behavior
        #pass input thru first linear layer
        x = F.relu(self.fcin(x))
        #reshape to to 4d tensor to fit into conv layers
        x = x.view(-1,self.conv_dim*16,1,1)
        #pass thru conv layers
        x = self.dropout1d(x)
        x = F.relu(self.deconv1(x))
        x = self.dropout2d(x)
        x = self.deconv2(x)
        x = F.relu(self.in2(x))
        x = self.dropout2d(x)
        x = self.deconv3(x)
        x = F.relu(self.in3(x))
        x = self.dropout2d(x)
        x = self.deconv4(x)
        x = F.relu(self.in4(x))
        x = self.dropout2d(x)
        #pass final output thru tan activation function
        out = F.tanh(self.deconv5(x))
        

        return out

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
tests.test_generator(Generator)
Tests Passed

Initialize the weights of your networks

To help your models converge, you should initialize the weights of the convolutional and linear layers in your model. From reading the original DCGAN paper, they say:

All weights were initialized from a zero-centered Normal distribution with standard deviation 0.02.

So, your next task will be to define a weight initialization function that does just this!

You can refer back to the lesson on weight initialization or even consult existing model code, such as that from the networks.py file in CycleGAN Github repository to help you complete this function.

Exercise: Complete the weight initialization function

  • This should initialize only convolutional and linear layers
  • Initialize the weights to a normal distribution, centered around 0, with a standard deviation of 0.02.
  • The bias terms, if they exist, may be left alone or set to 0.
In [31]:
def weights_init_normal(m):
    """
    Applies initial weights to certain layers in a model .
    The weights are taken from a normal distribution 
    with mean = 0, std dev = 0.02.
    :param m: A module or layer in a network    
    """
    # classname will be something like:
    # `Conv`, `BatchNorm2d`, `Linear`, etc.
    classname = m.__class__.__name__
    
    # TODO: Apply initial weights to convolutional and linear layers
    #class is a conv layer or linear layer, adjust weights
    if 'Conv' in classname or 'Linear' in classname:
        m.weight.data.uniform_(0.0, 0.02)
        m.bias.data.fill_(0)
    

Build complete network

Define your models' hyperparameters and instantiate the discriminator and generator from the classes defined above. Make sure you've passed in the correct input arguments.

In [32]:
"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
def build_network(d_conv_dim, g_conv_dim, z_size):
    # define discriminator and generator
    D = Discriminator(d_conv_dim)
    G = Generator(z_size=z_size, conv_dim=g_conv_dim)
    
    # initialize model weights
    D.apply(weights_init_normal)
    G.apply(weights_init_normal)

    print(D)
    print()
    print(G)
    
    return D, G

Exercise: Define model hyperparameters

In [33]:
# Define model hyperparams
d_conv_dim = 128
g_conv_dim = 128
z_size = 100

"""
DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE
"""
D, G = build_network(d_conv_dim, g_conv_dim, z_size)
Discriminator(
  (conv1): Conv2d(3, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (conv2): Conv2d(128, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (in2): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
  (conv3): Conv2d(256, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (in3): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
  (fcout): Linear(in_features=8192, out_features=1, bias=True)
  (dropout): Dropout2d(p=0.3, inplace=False)
  (dropout1d): Dropout(p=0.3, inplace=False)
)

Generator(
  (fcin): Linear(in_features=100, out_features=2048, bias=True)
  (deconv1): ConvTranspose2d(2048, 1024, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (in1): InstanceNorm2d(1024, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
  (deconv2): ConvTranspose2d(1024, 512, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (in2): InstanceNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
  (deconv3): ConvTranspose2d(512, 256, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (in3): InstanceNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
  (deconv4): ConvTranspose2d(256, 128, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (in4): InstanceNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=False)
  (deconv5): ConvTranspose2d(128, 3, kernel_size=(4, 4), stride=(2, 2), padding=(1, 1))
  (dropout1d): Dropout(p=0.3, inplace=False)
  (dropout2d): Dropout2d(p=0.3, inplace=False)
)

Training on GPU

Check if you can train on GPU. Here, we'll set this as a boolean variable train_on_gpu. Later, you'll be responsible for making sure that

  • Models,
  • Model inputs, and
  • Loss function arguments

Are moved to GPU, where appropriate.

In [14]:
"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
import torch

# Check for a GPU
train_on_gpu = torch.cuda.is_available()
if not train_on_gpu:
    print('No GPU found. Please use a GPU to train your neural network.')
else:
    print('Training on GPU!')
Training on GPU!

Discriminator and Generator Losses

Now we need to calculate the losses for both types of adversarial networks.

Discriminator Losses

  • For the discriminator, the total loss is the sum of the losses for real and fake images, d_loss = d_real_loss + d_fake_loss.
  • Remember that we want the discriminator to output 1 for real images and 0 for fake images, so we need to set up the losses to reflect that.

Generator Loss

The generator loss will look similar only with flipped labels. The generator's goal is to get the discriminator to think its generated images are real.

Exercise: Complete real and fake loss functions

You may choose to use either cross entropy or a least squares error loss to complete the following real_loss and fake_loss functions.

In [15]:
def real_loss(D_out, smooth=False):
    # compare logits to real labels
    # smooth labels if smooth=True
    #get batch size of output
    batch_size = D_out.size(0)
    if smooth:
       #create row of torch 1's as pos labels
        labels = torch.ones(batch_size) * 0.9
    else:
          #create row of torch 1's as pos labels
        labels = torch.ones(batch_size)
   #define loss method
    labels = labels.cuda()
    criterion = nn.BCEWithLogitsLoss()
    #return loss, squeezing out any empty dims
    return criterion(D_out.squeeze(),labels.squeeze())

def fake_loss(D_out):
    # compare logits to fake labels
    #get batch size
    batch_size = D_out.size(0)
    #make row of zeros as neg label
    labels = torch.zeros(batch_size)
    labels = labels.cuda()
    #define loss method
    criterion = nn.BCEWithLogitsLoss()
    #return loss, squeezing out any empty dims
    return criterion(D_out.squeeze(),labels.squeeze())

Optimizers

Exercise: Define optimizers for your Discriminator (D) and Generator (G)

Define optimizers for your models with appropriate hyperparameters.

In [16]:
import torch.optim as optim
# Create optimizers for the discriminator D and generator G
#use default lr for Adam
d_optimizer = optim.Adam(D.parameters(),lr= 0.0005 ,betas=[0.3,0.999])
g_optimizer = optim.Adam(G.parameters(),lr = 0.0005 , betas=[0.3,0.999])
In [17]:
def view_samples(epoch, samples):
    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
    for ax, img in zip(axes.flatten(), samples[epoch]):
        img = img.detach().cpu().numpy()
        img = np.transpose(img, (1, 2, 0))
        img = ((img + 1)*255 / (2)).astype(np.uint8)
        ax.xaxis.set_visible(False)
        ax.yaxis.set_visible(False)
        im = ax.imshow(img.reshape((32,32,3)))

Training

Training will involve alternating between training the discriminator and the generator. You'll use your functions real_loss and fake_loss to help you calculate the discriminator losses.

  • You should train the discriminator by alternating on real and fake images
  • Then the generator, which tries to trick the discriminator and should have an opposing loss function

Saving Samples

You've been given some code to print out some loss statistics and save some generated "fake" samples.

Exercise: Complete the training function

Keep in mind that, if you've moved your models to GPU, you'll also have to move any model inputs to GPU.

In [18]:
def train(D, G, n_epochs, print_every=50):
    '''Trains adversarial networks for some number of epochs
       param, D: the discriminator network
       param, G: the generator network
       param, n_epochs: number of epochs to train for
       param, print_every: when to print and record the models' losses
       return: D and G losses'''

    # move models to GPU
    if train_on_gpu:
        D.cuda()
        G.cuda()

    # keep track of loss and generated, "fake" samples
    samples = []
    losses = []

    # Get some fixed data for sampling. These are images that are held
    # constant throughout training, and allow us to inspect the model's performance
    sample_size=16
    fixed_z = np.random.uniform(-1, 1, size=(sample_size, z_size))
    fixed_z = torch.from_numpy(fixed_z).float()
    # move z to GPU if available
    if train_on_gpu:
        fixed_z = fixed_z.cuda()

    # epoch training loop
    for epoch in range(n_epochs):

        # batch training loop
        for batch_i, (real_images, _) in enumerate(celeba_train_loader):

            batch_size = real_images.size(0)
            real_images = scale(real_images).cuda()

            # ===============================================
            #         YOUR CODE HERE: TRAIN THE NETWORKS
            # ===============================================
            
            # 1. Train the discriminator on real and fake images
            #clear optimizer gradients
            d_optimizer.zero_grad()
            # Generate latent vector
            z = np.random.uniform(-1, 1, size=(batch_size, z_size))
            z = torch.from_numpy(z).float().cuda()
            #generate fake img
            fake_img = G(z)
            #put real image thru discrimator
            real_img_d= D(real_images)
            #Put fake image thru discrimator
            fake_img_d = D(fake_img)
            #generate real loss with smoothing
            real_l = real_loss(real_img_d,smooth=True)
            #generate fake loss
            fake_l = fake_loss(fake_img_d)
            #sum up losses
            d_loss = real_l + fake_l
            #backprop thru network
            d_loss.backward()
            #take optimization step
            d_optimizer.step()

            # 2. Train the generator with an adversarial loss
            g_optimizer.zero_grad()
            
            # Generate latent vector
            z = np.random.uniform(-1, 1, size=(batch_size, z_size))
            z = torch.from_numpy(z).float().cuda()
            #generate fake img
            fake_img = G(z)
            #put fake image thru discrimanator
            fake_img_d = D(fake_img)
            #generate real loss on fake_img
            real_l = real_loss(fake_img_d,smooth=True)
            #sum up losses
            g_loss = real_l
            #backprop thru network
            g_loss.backward()
            #take optimization step
            g_optimizer.step()
            
            
            # ===============================================
            #              END OF YOUR CODE
            # ===============================================

            # Print some loss stats
            if batch_i % print_every == 0:
                # append discriminator loss and generator loss
                losses.append((d_loss.item(), g_loss.item()))
                # print discriminator and generator loss
                print('Epoch [{:5d}/{:5d}] | d_loss: {:6.4f} | g_loss: {:6.4f}'.format(
                        epoch+1, n_epochs, d_loss.item(), g_loss.item()))


        ## AFTER EACH EPOCH##    
        # this code assumes your generator is named G, feel free to change the name
        # generate and save sample, fake images
        G.eval() # for generating samples
        samples_z = G(fixed_z)
        samples.append(samples_z)
        G.train() # back to training mode
        #view images
        view_samples(epoch,samples)
    # Save training generator samples
    with open('train_samples.pkl', 'wb') as f:
        pkl.dump(samples, f)
    
    # finally return losses
    return losses

Set your number of training epochs and train your GAN!

In [24]:
# set number of epochs 
n_epochs = 25

"""
DON'T MODIFY ANYTHING IN THIS CELL
"""
# call training function
losses = train(D, G, n_epochs=n_epochs)
91 | g_loss: 3.3139
Epoch [   23/   25] | d_loss: 0.8271 | g_loss: 2.3705
Epoch [   23/   25] | d_loss: 0.6774 | g_loss: 3.9828
Epoch [   23/   25] | d_loss: 0.6154 | g_loss: 3.5293
Epoch [   23/   25] | d_loss: 0.6429 | g_loss: 1.9354
Epoch [   23/   25] | d_loss: 0.6624 | g_loss: 3.6539
Epoch [   23/   25] | d_loss: 0.6927 | g_loss: 2.1527
Epoch [   23/   25] | d_loss: 0.6473 | g_loss: 3.3234
Epoch [   23/   25] | d_loss: 0.7114 | g_loss: 3.9641
Epoch [   23/   25] | d_loss: 0.7495 | g_loss: 2.4018
Epoch [   23/   25] | d_loss: 0.6515 | g_loss: 3.3457
Epoch [   23/   25] | d_loss: 0.5248 | g_loss: 3.2014
Epoch [   23/   25] | d_loss: 0.6208 | g_loss: 2.4882
Epoch [   23/   25] | d_loss: 0.7080 | g_loss: 2.0384
Epoch [   23/   25] | d_loss: 1.0932 | g_loss: 2.3856
Epoch [   23/   25] | d_loss: 0.6949 | g_loss: 3.0669
Epoch [   23/   25] | d_loss: 0.5926 | g_loss: 3.0898
Epoch [   23/   25] | d_loss: 0.8540 | g_loss: 2.6641
Epoch [   23/   25] | d_loss: 0.5944 | g_loss: 2.5515
Epoch [   23/   25] | d_loss: 0.5942 | g_loss: 3.0917
Epoch [   23/   25] | d_loss: 0.6964 | g_loss: 2.7776
Epoch [   23/   25] | d_loss: 0.6921 | g_loss: 3.2980
Epoch [   23/   25] | d_loss: 0.7931 | g_loss: 3.2218
Epoch [   23/   25] | d_loss: 0.9040 | g_loss: 2.2274
Epoch [   23/   25] | d_loss: 0.6570 | g_loss: 3.5472
Epoch [   23/   25] | d_loss: 0.6678 | g_loss: 2.6792
Epoch [   23/   25] | d_loss: 0.7407 | g_loss: 2.6143
Epoch [   23/   25] | d_loss: 0.6429 | g_loss: 2.6784
Epoch [   23/   25] | d_loss: 1.0157 | g_loss: 2.0656
Epoch [   23/   25] | d_loss: 0.7577 | g_loss: 3.4089
Epoch [   23/   25] | d_loss: 1.2774 | g_loss: 1.9399
Epoch [   23/   25] | d_loss: 0.6419 | g_loss: 2.7434
Epoch [   23/   25] | d_loss: 0.8026 | g_loss: 3.6703
Epoch [   23/   25] | d_loss: 0.7281 | g_loss: 2.5514
Epoch [   23/   25] | d_loss: 0.9306 | g_loss: 2.3284
Epoch [   23/   25] | d_loss: 0.6962 | g_loss: 1.8853
Epoch [   23/   25] | d_loss: 0.8197 | g_loss: 2.6040
Epoch [   23/   25] | d_loss: 0.7974 | g_loss: 2.1022
Epoch [   23/   25] | d_loss: 0.6519 | g_loss: 2.6595
Epoch [   23/   25] | d_loss: 0.6039 | g_loss: 3.6534
Epoch [   23/   25] | d_loss: 0.6571 | g_loss: 2.8807
Epoch [   23/   25] | d_loss: 0.7298 | g_loss: 2.8011
Epoch [   23/   25] | d_loss: 0.6002 | g_loss: 3.6899
Epoch [   23/   25] | d_loss: 0.7121 | g_loss: 1.6283
Epoch [   23/   25] | d_loss: 0.6345 | g_loss: 3.3329
Epoch [   23/   25] | d_loss: 0.8444 | g_loss: 4.0887
Epoch [   23/   25] | d_loss: 0.6562 | g_loss: 2.5528
Epoch [   23/   25] | d_loss: 0.6301 | g_loss: 2.7596
Epoch [   23/   25] | d_loss: 0.9251 | g_loss: 2.2982
Epoch [   23/   25] | d_loss: 0.8071 | g_loss: 3.5871
Epoch [   23/   25] | d_loss: 0.7090 | g_loss: 2.3626
Epoch [   23/   25] | d_loss: 0.6012 | g_loss: 3.2512
Epoch [   23/   25] | d_loss: 0.6546 | g_loss: 2.9446
Epoch [   23/   25] | d_loss: 0.8098 | g_loss: 3.6561
Epoch [   23/   25] | d_loss: 0.7167 | g_loss: 4.4880
Epoch [   23/   25] | d_loss: 0.7717 | g_loss: 2.5280
Epoch [   23/   25] | d_loss: 0.6940 | g_loss: 3.0527
Epoch [   23/   25] | d_loss: 0.7319 | g_loss: 2.7560
Epoch [   23/   25] | d_loss: 0.8673 | g_loss: 2.3672
Epoch [   23/   25] | d_loss: 0.8620 | g_loss: 3.6917
Epoch [   23/   25] | d_loss: 0.5886 | g_loss: 2.8439
Epoch [   23/   25] | d_loss: 0.7430 | g_loss: 3.0483
Epoch [   23/   25] | d_loss: 0.8324 | g_loss: 3.3912
Epoch [   23/   25] | d_loss: 0.8014 | g_loss: 2.6053
Epoch [   23/   25] | d_loss: 0.6631 | g_loss: 3.2493
Epoch [   23/   25] | d_loss: 0.7809 | g_loss: 2.3874
Epoch [   23/   25] | d_loss: 0.5809 | g_loss: 3.3034
Epoch [   23/   25] | d_loss: 0.7604 | g_loss: 2.9151
Epoch [   23/   25] | d_loss: 0.8565 | g_loss: 2.6566
Epoch [   23/   25] | d_loss: 0.6569 | g_loss: 2.2962
Epoch [   23/   25] | d_loss: 0.6675 | g_loss: 2.7566
Epoch [   23/   25] | d_loss: 0.8406 | g_loss: 3.8753
Epoch [   23/   25] | d_loss: 0.5575 | g_loss: 2.8318
Epoch [   23/   25] | d_loss: 0.7171 | g_loss: 2.7905
Epoch [   23/   25] | d_loss: 0.6622 | g_loss: 2.8631
Epoch [   23/   25] | d_loss: 0.8586 | g_loss: 3.5156
Epoch [   23/   25] | d_loss: 0.7272 | g_loss: 2.7113
Epoch [   23/   25] | d_loss: 0.7189 | g_loss: 2.6473
Epoch [   23/   25] | d_loss: 0.7093 | g_loss: 2.2259
Epoch [   23/   25] | d_loss: 1.1490 | g_loss: 1.2090
Epoch [   23/   25] | d_loss: 0.7721 | g_loss: 3.9108
Epoch [   23/   25] | d_loss: 0.6555 | g_loss: 2.7979
Epoch [   23/   25] | d_loss: 0.7593 | g_loss: 3.3639
Epoch [   23/   25] | d_loss: 0.7393 | g_loss: 2.2409
Epoch [   23/   25] | d_loss: 0.6355 | g_loss: 2.9553
Epoch [   23/   25] | d_loss: 0.6938 | g_loss: 3.7523
Epoch [   23/   25] | d_loss: 1.0839 | g_loss: 1.4716
Epoch [   23/   25] | d_loss: 0.5887 | g_loss: 3.4754
Epoch [   23/   25] | d_loss: 0.8273 | g_loss: 2.2005
Epoch [   23/   25] | d_loss: 0.6971 | g_loss: 2.6081
Epoch [   23/   25] | d_loss: 0.6858 | g_loss: 3.0623
Epoch [   23/   25] | d_loss: 0.6794 | g_loss: 2.4338
Epoch [   23/   25] | d_loss: 0.7320 | g_loss: 2.7381
Epoch [   23/   25] | d_loss: 0.8720 | g_loss: 2.7008
Epoch [   23/   25] | d_loss: 0.6411 | g_loss: 3.1399
Epoch [   23/   25] | d_loss: 0.7646 | g_loss: 2.7228
Epoch [   23/   25] | d_loss: 0.7640 | g_loss: 2.5385
Epoch [   23/   25] | d_loss: 0.8332 | g_loss: 2.4083
Epoch [   23/   25] | d_loss: 0.7607 | g_loss: 2.9830
Epoch [   23/   25] | d_loss: 0.7070 | g_loss: 3.3521
Epoch [   23/   25] | d_loss: 0.9982 | g_loss: 1.8707
Epoch [   23/   25] | d_loss: 0.7710 | g_loss: 2.7405
Epoch [   23/   25] | d_loss: 0.7422 | g_loss: 2.9746
Epoch [   23/   25] | d_loss: 0.7104 | g_loss: 2.3385
Epoch [   23/   25] | d_loss: 0.8274 | g_loss: 2.8986
Epoch [   23/   25] | d_loss: 0.6876 | g_loss: 2.5551
Epoch [   23/   25] | d_loss: 0.8698 | g_loss: 2.6386
Epoch [   23/   25] | d_loss: 0.6772 | g_loss: 3.0691
Epoch [   23/   25] | d_loss: 0.6989 | g_loss: 2.4885
Epoch [   23/   25] | d_loss: 0.6511 | g_loss: 2.4896
Epoch [   23/   25] | d_loss: 0.7251 | g_loss: 3.3835
Epoch [   23/   25] | d_loss: 0.5760 | g_loss: 3.7177
Epoch [   23/   25] | d_loss: 0.8664 | g_loss: 2.7679
Epoch [   23/   25] | d_loss: 0.6552 | g_loss: 2.7303
Epoch [   23/   25] | d_loss: 0.8721 | g_loss: 2.4665
Epoch [   23/   25] | d_loss: 0.8962 | g_loss: 2.3045
Epoch [   23/   25] | d_loss: 0.6454 | g_loss: 3.7210
Epoch [   24/   25] | d_loss: 0.8863 | g_loss: 2.1890
Epoch [   24/   25] | d_loss: 0.7640 | g_loss: 2.0908
Epoch [   24/   25] | d_loss: 0.6888 | g_loss: 2.5088
Epoch [   24/   25] | d_loss: 0.9517 | g_loss: 2.5979
Epoch [   24/   25] | d_loss: 0.7725 | g_loss: 2.6666
Epoch [   24/   25] | d_loss: 0.6100 | g_loss: 3.0674
Epoch [   24/   25] | d_loss: 0.8191 | g_loss: 3.0604
Epoch [   24/   25] | d_loss: 0.9136 | g_loss: 3.6077
Epoch [   24/   25] | d_loss: 0.7906 | g_loss: 2.3547
Epoch [   24/   25] | d_loss: 0.6671 | g_loss: 2.2943
Epoch [   24/   25] | d_loss: 0.6372 | g_loss: 3.3857
Epoch [   24/   25] | d_loss: 0.7969 | g_loss: 2.4727
Epoch [   24/   25] | d_loss: 0.8607 | g_loss: 2.1674
Epoch [   24/   25] | d_loss: 0.7800 | g_loss: 3.4819
Epoch [   24/   25] | d_loss: 0.7883 | g_loss: 3.0387
Epoch [   24/   25] | d_loss: 0.9171 | g_loss: 2.0087
Epoch [   24/   25] | d_loss: 0.6020 | g_loss: 3.3284
Epoch [   24/   25] | d_loss: 0.8942 | g_loss: 2.8417
Epoch [   24/   25] | d_loss: 0.9488 | g_loss: 2.0964
Epoch [   24/   25] | d_loss: 0.5610 | g_loss: 2.6171
Epoch [   24/   25] | d_loss: 0.6038 | g_loss: 3.9006
Epoch [   24/   25] | d_loss: 0.7967 | g_loss: 2.7388
Epoch [   24/   25] | d_loss: 0.7084 | g_loss: 4.1670
Epoch [   24/   25] | d_loss: 0.6427 | g_loss: 2.3520
Epoch [   24/   25] | d_loss: 0.6605 | g_loss: 3.7029
Epoch [   24/   25] | d_loss: 0.8508 | g_loss: 3.7908
Epoch [   24/   25] | d_loss: 0.6130 | g_loss: 3.4387
Epoch [   24/   25] | d_loss: 0.7002 | g_loss: 3.4517
Epoch [   24/   25] | d_loss: 0.7745 | g_loss: 2.1264
Epoch [   24/   25] | d_loss: 0.7740 | g_loss: 3.2669
Epoch [   24/   25] | d_loss: 0.6102 | g_loss: 2.8598
Epoch [   24/   25] | d_loss: 0.8538 | g_loss: 3.8187
Epoch [   24/   25] | d_loss: 1.0667 | g_loss: 3.4521
Epoch [   24/   25] | d_loss: 0.6917 | g_loss: 2.8198
Epoch [   24/   25] | d_loss: 0.5991 | g_loss: 2.8554
Epoch [   24/   25] | d_loss: 0.6130 | g_loss: 2.9516
Epoch [   24/   25] | d_loss: 0.6079 | g_loss: 3.3347
Epoch [   24/   25] | d_loss: 0.8667 | g_loss: 2.0806
Epoch [   24/   25] | d_loss: 0.6725 | g_loss: 3.2518
Epoch [   24/   25] | d_loss: 0.6516 | g_loss: 2.4194
Epoch [   24/   25] | d_loss: 0.7923 | g_loss: 3.4272
Epoch [   24/   25] | d_loss: 0.5941 | g_loss: 2.5687
Epoch [   24/   25] | d_loss: 0.6924 | g_loss: 3.4127
Epoch [   24/   25] | d_loss: 0.6034 | g_loss: 3.0472
Epoch [   24/   25] | d_loss: 0.6342 | g_loss: 3.3871
Epoch [   24/   25] | d_loss: 0.7204 | g_loss: 3.2560
Epoch [   24/   25] | d_loss: 0.6350 | g_loss: 3.5575
Epoch [   24/   25] | d_loss: 1.0800 | g_loss: 2.4429
Epoch [   24/   25] | d_loss: 0.6513 | g_loss: 1.8876
Epoch [   24/   25] | d_loss: 0.7670 | g_loss: 2.3479
Epoch [   24/   25] | d_loss: 0.8294 | g_loss: 3.0366
Epoch [   24/   25] | d_loss: 0.6918 | g_loss: 3.2355
Epoch [   24/   25] | d_loss: 0.8780 | g_loss: 4.0159
Epoch [   24/   25] | d_loss: 0.7647 | g_loss: 2.4254
Epoch [   24/   25] | d_loss: 0.6671 | g_loss: 4.1209
Epoch [   24/   25] | d_loss: 0.7898 | g_loss: 2.9575
Epoch [   24/   25] | d_loss: 0.7385 | g_loss: 3.4017
Epoch [   24/   25] | d_loss: 0.7176 | g_loss: 2.3045
Epoch [   24/   25] | d_loss: 0.6754 | g_loss: 2.3104
Epoch [   24/   25] | d_loss: 0.6516 | g_loss: 3.5727
Epoch [   24/   25] | d_loss: 0.7303 | g_loss: 2.8149
Epoch [   24/   25] | d_loss: 0.7412 | g_loss: 2.7786
Epoch [   24/   25] | d_loss: 0.7843 | g_loss: 4.1804
Epoch [   24/   25] | d_loss: 0.7438 | g_loss: 3.3737
Epoch [   24/   25] | d_loss: 0.6803 | g_loss: 2.7825
Epoch [   24/   25] | d_loss: 0.7424 | g_loss: 3.0492
Epoch [   24/   25] | d_loss: 0.5380 | g_loss: 2.5431
Epoch [   24/   25] | d_loss: 0.6281 | g_loss: 2.5309
Epoch [   24/   25] | d_loss: 0.7050 | g_loss: 3.6319
Epoch [   24/   25] | d_loss: 0.8625 | g_loss: 2.1781
Epoch [   24/   25] | d_loss: 0.6996 | g_loss: 2.7343
Epoch [   24/   25] | d_loss: 0.6555 | g_loss: 2.1947
Epoch [   24/   25] | d_loss: 0.8162 | g_loss: 1.7668
Epoch [   24/   25] | d_loss: 0.6925 | g_loss: 2.2597
Epoch [   24/   25] | d_loss: 0.6955 | g_loss: 2.9339
Epoch [   24/   25] | d_loss: 0.7047 | g_loss: 1.9743
Epoch [   24/   25] | d_loss: 0.6498 | g_loss: 2.8349
Epoch [   24/   25] | d_loss: 0.6675 | g_loss: 4.1025
Epoch [   24/   25] | d_loss: 0.7948 | g_loss: 3.0251
Epoch [   24/   25] | d_loss: 0.6480 | g_loss: 3.2551
Epoch [   24/   25] | d_loss: 0.7299 | g_loss: 3.2523
Epoch [   24/   25] | d_loss: 0.7240 | g_loss: 2.4189
Epoch [   24/   25] | d_loss: 0.7261 | g_loss: 2.3208
Epoch [   24/   25] | d_loss: 0.8340 | g_loss: 3.4801
Epoch [   24/   25] | d_loss: 0.8675 | g_loss: 2.7920
Epoch [   24/   25] | d_loss: 0.6739 | g_loss: 2.4553
Epoch [   24/   25] | d_loss: 0.6307 | g_loss: 2.5821
Epoch [   24/   25] | d_loss: 0.6065 | g_loss: 2.4899
Epoch [   24/   25] | d_loss: 0.6399 | g_loss: 2.6458
Epoch [   24/   25] | d_loss: 0.5641 | g_loss: 2.1804
Epoch [   24/   25] | d_loss: 0.7582 | g_loss: 3.1788
Epoch [   24/   25] | d_loss: 0.7370 | g_loss: 2.8298
Epoch [   24/   25] | d_loss: 0.7526 | g_loss: 3.1622
Epoch [   24/   25] | d_loss: 0.8043 | g_loss: 2.7136
Epoch [   24/   25] | d_loss: 0.7053 | g_loss: 3.4769
Epoch [   24/   25] | d_loss: 0.7694 | g_loss: 1.9023
Epoch [   24/   25] | d_loss: 0.7285 | g_loss: 2.2513
Epoch [   24/   25] | d_loss: 0.6661 | g_loss: 3.0787
Epoch [   24/   25] | d_loss: 0.9196 | g_loss: 2.8729
Epoch [   24/   25] | d_loss: 0.7543 | g_loss: 4.2113
Epoch [   24/   25] | d_loss: 0.6726 | g_loss: 2.7950
Epoch [   24/   25] | d_loss: 0.8140 | g_loss: 2.4738
Epoch [   24/   25] | d_loss: 0.8691 | g_loss: 3.9789
Epoch [   24/   25] | d_loss: 0.7719 | g_loss: 3.3817
Epoch [   24/   25] | d_loss: 0.7236 | g_loss: 2.3863
Epoch [   24/   25] | d_loss: 1.1627 | g_loss: 2.1361
Epoch [   24/   25] | d_loss: 0.6300 | g_loss: 3.7966
Epoch [   24/   25] | d_loss: 0.6558 | g_loss: 2.9494
Epoch [   24/   25] | d_loss: 0.8627 | g_loss: 3.8006
Epoch [   24/   25] | d_loss: 0.6685 | g_loss: 3.3631
Epoch [   24/   25] | d_loss: 0.6965 | g_loss: 3.9370
Epoch [   24/   25] | d_loss: 0.6265 | g_loss: 2.8789
Epoch [   24/   25] | d_loss: 0.8099 | g_loss: 3.3281
Epoch [   24/   25] | d_loss: 0.7303 | g_loss: 2.4307
Epoch [   24/   25] | d_loss: 1.0231 | g_loss: 1.5609
Epoch [   24/   25] | d_loss: 0.8778 | g_loss: 3.7933
Epoch [   24/   25] | d_loss: 0.7060 | g_loss: 2.7081
Epoch [   24/   25] | d_loss: 0.5850 | g_loss: 2.5109
Epoch [   24/   25] | d_loss: 0.7085 | g_loss: 2.0503
Epoch [   24/   25] | d_loss: 0.6751 | g_loss: 2.3532
Epoch [   24/   25] | d_loss: 0.6539 | g_loss: 1.8497
Epoch [   24/   25] | d_loss: 0.8062 | g_loss: 3.3628
Epoch [   24/   25] | d_loss: 0.9097 | g_loss: 1.8346
Epoch [   24/   25] | d_loss: 0.7744 | g_loss: 1.9844
Epoch [   24/   25] | d_loss: 0.6824 | g_loss: 4.3813
Epoch [   24/   25] | d_loss: 0.6123 | g_loss: 2.8853
Epoch [   24/   25] | d_loss: 0.6356 | g_loss: 2.5812
Epoch [   25/   25] | d_loss: 0.6553 | g_loss: 3.3381
Epoch [   25/   25] | d_loss: 0.7488 | g_loss: 1.8808
Epoch [   25/   25] | d_loss: 0.7573 | g_loss: 2.3552
Epoch [   25/   25] | d_loss: 0.8523 | g_loss: 2.6505
Epoch [   25/   25] | d_loss: 0.6893 | g_loss: 1.6463
Epoch [   25/   25] | d_loss: 0.6827 | g_loss: 2.7033
Epoch [   25/   25] | d_loss: 1.3370 | g_loss: 1.7372
Epoch [   25/   25] | d_loss: 0.9016 | g_loss: 2.8308
Epoch [   25/   25] | d_loss: 0.7652 | g_loss: 3.0515
Epoch [   25/   25] | d_loss: 0.5393 | g_loss: 2.8627
Epoch [   25/   25] | d_loss: 0.6323 | g_loss: 2.4705
Epoch [   25/   25] | d_loss: 0.5935 | g_loss: 2.0151
Epoch [   25/   25] | d_loss: 0.7899 | g_loss: 2.9506
Epoch [   25/   25] | d_loss: 0.6834 | g_loss: 2.8895
Epoch [   25/   25] | d_loss: 0.7972 | g_loss: 3.2527
Epoch [   25/   25] | d_loss: 0.7265 | g_loss: 3.1797
Epoch [   25/   25] | d_loss: 0.6774 | g_loss: 3.2523
Epoch [   25/   25] | d_loss: 0.6469 | g_loss: 3.9228
Epoch [   25/   25] | d_loss: 0.5964 | g_loss: 3.8591
Epoch [   25/   25] | d_loss: 0.7537 | g_loss: 3.4838
Epoch [   25/   25] | d_loss: 0.7276 | g_loss: 3.1574
Epoch [   25/   25] | d_loss: 0.5937 | g_loss: 2.1671
Epoch [   25/   25] | d_loss: 0.6430 | g_loss: 2.2127
Epoch [   25/   25] | d_loss: 0.6114 | g_loss: 2.9096
Epoch [   25/   25] | d_loss: 0.7155 | g_loss: 3.3325
Epoch [   25/   25] | d_loss: 0.7188 | g_loss: 3.9043
Epoch [   25/   25] | d_loss: 0.9421 | g_loss: 2.6251
Epoch [   25/   25] | d_loss: 0.9739 | g_loss: 4.0249
Epoch [   25/   25] | d_loss: 0.8143 | g_loss: 2.6846
Epoch [   25/   25] | d_loss: 0.7255 | g_loss: 2.4260
Epoch [   25/   25] | d_loss: 0.5981 | g_loss: 2.7342
Epoch [   25/   25] | d_loss: 0.8308 | g_loss: 2.9766
Epoch [   25/   25] | d_loss: 0.8463 | g_loss: 3.2330
Epoch [   25/   25] | d_loss: 0.5945 | g_loss: 2.9258
Epoch [   25/   25] | d_loss: 0.8228 | g_loss: 3.7213
Epoch [   25/   25] | d_loss: 0.7973 | g_loss: 2.7125
Epoch [   25/   25] | d_loss: 0.7173 | g_loss: 3.4103
Epoch [   25/   25] | d_loss: 0.7257 | g_loss: 2.5495
Epoch [   25/   25] | d_loss: 0.7310 | g_loss: 3.0423
Epoch [   25/   25] | d_loss: 0.6741 | g_loss: 2.3517
Epoch [   25/   25] | d_loss: 0.6174 | g_loss: 2.2086
Epoch [   25/   25] | d_loss: 0.8185 | g_loss: 3.0420
Epoch [   25/   25] | d_loss: 0.6476 | g_loss: 3.7654
Epoch [   25/   25] | d_loss: 0.7344 | g_loss: 3.7812
Epoch [   25/   25] | d_loss: 0.8290 | g_loss: 2.9445
Epoch [   25/   25] | d_loss: 0.7781 | g_loss: 1.8952
Epoch [   25/   25] | d_loss: 0.7782 | g_loss: 3.3541
Epoch [   25/   25] | d_loss: 0.8078 | g_loss: 3.0709
Epoch [   25/   25] | d_loss: 0.7771 | g_loss: 2.5601
Epoch [   25/   25] | d_loss: 0.5734 | g_loss: 2.7742
Epoch [   25/   25] | d_loss: 0.8147 | g_loss: 3.3424
Epoch [   25/   25] | d_loss: 0.6827 | g_loss: 2.4547
Epoch [   25/   25] | d_loss: 0.9021 | g_loss: 1.7479
Epoch [   25/   25] | d_loss: 0.8628 | g_loss: 4.0704
Epoch [   25/   25] | d_loss: 0.6301 | g_loss: 3.0490
Epoch [   25/   25] | d_loss: 0.6647 | g_loss: 3.1720
Epoch [   25/   25] | d_loss: 1.1093 | g_loss: 4.0813
Epoch [   25/   25] | d_loss: 0.6544 | g_loss: 3.2097
Epoch [   25/   25] | d_loss: 0.5811 | g_loss: 3.6678
Epoch [   25/   25] | d_loss: 0.9938 | g_loss: 2.7488
Epoch [   25/   25] | d_loss: 0.7218 | g_loss: 1.9988
Epoch [   25/   25] | d_loss: 0.6914 | g_loss: 3.0478
Epoch [   25/   25] | d_loss: 0.5537 | g_loss: 2.9855
Epoch [   25/   25] | d_loss: 0.7102 | g_loss: 2.3534
Epoch [   25/   25] | d_loss: 0.6466 | g_loss: 2.4022
Epoch [   25/   25] | d_loss: 0.8738 | g_loss: 2.1781
Epoch [   25/   25] | d_loss: 0.9081 | g_loss: 2.6967
Epoch [   25/   25] | d_loss: 0.5928 | g_loss: 2.7840
Epoch [   25/   25] | d_loss: 0.6951 | g_loss: 2.8386
Epoch [   25/   25] | d_loss: 0.6811 | g_loss: 3.1811
Epoch [   25/   25] | d_loss: 0.7695 | g_loss: 2.7803
Epoch [   25/   25] | d_loss: 0.5619 | g_loss: 3.1387
Epoch [   25/   25] | d_loss: 0.7416 | g_loss: 2.6776
Epoch [   25/   25] | d_loss: 0.6589 | g_loss: 2.7698
Epoch [   25/   25] | d_loss: 0.7476 | g_loss: 1.9866
Epoch [   25/   25] | d_loss: 0.5714 | g_loss: 2.1343
Epoch [   25/   25] | d_loss: 0.6694 | g_loss: 3.0043
Epoch [   25/   25] | d_loss: 0.8032 | g_loss: 4.6421
Epoch [   25/   25] | d_loss: 0.5958 | g_loss: 3.8398
Epoch [   25/   25] | d_loss: 0.6533 | g_loss: 3.2500
Epoch [   25/   25] | d_loss: 0.7182 | g_loss: 3.6055
Epoch [   25/   25] | d_loss: 0.6971 | g_loss: 1.8618
Epoch [   25/   25] | d_loss: 0.6192 | g_loss: 2.4879
Epoch [   25/   25] | d_loss: 0.5332 | g_loss: 3.4009
Epoch [   25/   25] | d_loss: 0.7567 | g_loss: 1.6840
Epoch [   25/   25] | d_loss: 0.5969 | g_loss: 2.6565
Epoch [   25/   25] | d_loss: 0.8803 | g_loss: 1.9252
Epoch [   25/   25] | d_loss: 0.6848 | g_loss: 3.2694
Epoch [   25/   25] | d_loss: 0.8270 | g_loss: 2.1507
Epoch [   25/   25] | d_loss: 0.7076 | g_loss: 2.2284
Epoch [   25/   25] | d_loss: 0.7557 | g_loss: 2.5048
Epoch [   25/   25] | d_loss: 0.6882 | g_loss: 2.1312
Epoch [   25/   25] | d_loss: 0.7739 | g_loss: 1.7674
Epoch [   25/   25] | d_loss: 0.6834 | g_loss: 2.9918
Epoch [   25/   25] | d_loss: 0.7021 | g_loss: 2.8338
Epoch [   25/   25] | d_loss: 0.6599 | g_loss: 3.2692
Epoch [   25/   25] | d_loss: 0.7637 | g_loss: 3.2916
Epoch [   25/   25] | d_loss: 0.7549 | g_loss: 2.2635
Epoch [   25/   25] | d_loss: 0.7615 | g_loss: 4.0226
Epoch [   25/   25] | d_loss: 0.7006 | g_loss: 2.9587
Epoch [   25/   25] | d_loss: 1.2540 | g_loss: 1.3562
Epoch [   25/   25] | d_loss: 0.6812 | g_loss: 4.4631
Epoch [   25/   25] | d_loss: 0.8064 | g_loss: 3.1700
Epoch [   25/   25] | d_loss: 0.6346 | g_loss: 2.5584
Epoch [   25/   25] | d_loss: 0.6291 | g_loss: 2.8338
Epoch [   25/   25] | d_loss: 0.7950 | g_loss: 4.1151
Epoch [   25/   25] | d_loss: 0.6258 | g_loss: 2.5840
Epoch [   25/   25] | d_loss: 0.8407 | g_loss: 3.1796
Epoch [   25/   25] | d_loss: 0.7961 | g_loss: 3.5587
Epoch [   25/   25] | d_loss: 0.6429 | g_loss: 2.2886
Epoch [   25/   25] | d_loss: 0.5826 | g_loss: 2.6498
Epoch [   25/   25] | d_loss: 0.7264 | g_loss: 3.2587
Epoch [   25/   25] | d_loss: 0.6470 | g_loss: 3.2067
Epoch [   25/   25] | d_loss: 0.6292 | g_loss: 2.7570
Epoch [   25/   25] | d_loss: 0.6492 | g_loss: 2.4264
Epoch [   25/   25] | d_loss: 0.6956 | g_loss: 2.8836
Epoch [   25/   25] | d_loss: 0.8308 | g_loss: 2.1632
Epoch [   25/   25] | d_loss: 0.6773 | g_loss: 3.0052
Epoch [   25/   25] | d_loss: 0.7063 | g_loss: 3.3837
Epoch [   25/   25] | d_loss: 0.6521 | g_loss: 2.9392
Epoch [   25/   25] | d_loss: 0.5714 | g_loss: 2.2149
Epoch [   25/   25] | d_loss: 0.5955 | g_loss: 3.3351
Epoch [   25/   25] | d_loss: 0.7096 | g_loss: 2.6810
Epoch [   25/   25] | d_loss: 0.6974 | g_loss: 2.9261
Epoch [   25/   25] | d_loss: 0.9359 | g_loss: 2.2160
Epoch [   25/   25] | d_loss: 0.7545 | g_loss: 3.3156
Epoch [   25/   25] | d_loss: 0.6313 | g_loss: 2.2972

Training loss

Plot the training losses for the generator and discriminator, recorded after each epoch.

In [25]:
fig, ax = plt.subplots()
losses = np.array(losses)
plt.plot(losses.T[0], label='Discriminator', alpha=0.5)
plt.plot(losses.T[1], label='Generator', alpha=0.5)
plt.title("Training Losses")
plt.legend()
Out[25]:
<matplotlib.legend.Legend at 0x203ec2a1100>

Generator samples from training

View samples of images from the generator, and answer a question about the strengths and weaknesses of your trained models.

In [26]:
# helper function for viewing a list of passed in sample images
import pickle as pkl
import matplotlib.pyplot as plt
import numpy as np

def view_samples(epoch, samples):
    fig, axes = plt.subplots(figsize=(16,4), nrows=2, ncols=8, sharey=True, sharex=True)
    for ax, img in zip(axes.flatten(), samples[epoch]):
        img = img.detach().cpu().numpy()
        img = np.transpose(img, (1, 2, 0))
        img = ((img + 1)*255 / (2)).astype(np.uint8)
        ax.xaxis.set_visible(False)
        ax.yaxis.set_visible(False)
        im = ax.imshow(img.reshape((32,32,3)))
In [27]:
# Load samples from generator, taken while training
with open('train_samples.pkl', 'rb') as f:
    samples = pkl.load(f)
In [28]:
_ = view_samples(-1, samples)

Question: What do you notice about your generated samples and how might you improve this model?

When you answer this question, consider the following factors:

  • The dataset is biased; it is made of "celebrity" faces that are mostly white
  • Model size; larger models have the opportunity to learn more features in a data feature space
  • Optimization strategy; optimizers and number of epochs affect your final result

-My generated samples are all white, which caused at a data in the bias , making most of my generated set white aswell.

-My model was large, but perhaps not large enough to capture all the nuances of a human face.

-Perhaps a higher learning rate would have resulted in better generated faces.

Answer: (Write your answer in this cell)

Submitting This Project

When submitting this project, make sure to run all the cells before saving the notebook. Save the notebook file as "dlnd_face_generation.ipynb" and save it as a HTML file under "File" -> "Download as". Include the "problem_unittests.py" files in your submission.